of Technology Lecturer : Piotr Indyk 6 . 895 : Sketching , Streaming and Sub - linear Space Algorithms

consider the case where x is has exactly k + 1 nonzero entries. Then, Errk(x) = Err 2 k(x) = x (k+1), where x(k+1) represents the smallest of the k + 1 nonzero entries in x. Thus, the formula above implies that for such x, the LP finds an x∗ that is better than the best k-sparse approximation, so clearly x∗ cannot be k-sparse. In practice, it is often not important that x∗ be k-sparse. For example, in image processing, it may not be important that our reconstruction have exactly k non-zero frequency components, so reconstructing x to x∗ may be fine. However, if we do want to construct a k-sparse approximation x̃ to x, a simple application of the triangle inequality shows that setting x̃ to be the optimal k-sparse approximation1 to x∗ produces a good k-sparse approximation to x. Formally, let x′ denote the optimal k-sparse approximation to x. Then,